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Abstract. This study provides an initial statistical approach as an alternative model to detect 
the fraudulent practice estimation of national examination (NE). Trends, patterns, and 
relationships were analysed from the national examination (NE) scores and the national 
examination integrity index (NEII) on pre- and post-computer-based national examination 
(CBNE) of the 103,273 high school science students in Aceh Province. Looking at the trends 
and patterns of the pre-post CBNE implementation, descriptive statistics were used by 
comparing the average (Mean) of the NE, NEII, and the standard deviation of national 
examination (SDNE) in the group of regencies/cities. Picturing the relationship between the 
NE and NEII, linear regression and curve estimation was applied. The result shows that there 
are striking differences and gradual changes between the pre-post CBNE’s implementation in 
each regency/city. In the span of three years (2015-2017), a downward trend showed in the 
NE’s scores along with the implementation of CBNE which increased the NEII’s value. In 
the pre-CBNE period (2015), a striking pattern emerged in the form of low NEI, high NE, and 
low SDNE which indicated the existence of fraudulent practice estimation of the national 
examination. 


1. Introduction 

The national examination (NE) is a big issue in Indonesia education. The implementation involved 
millions of students, teaching staffs, high budget [1], with the prolonged debate [2]. Due to its risks, 
such an exam is categorized as a high-stake test. Indonesia was at the 2™ place of the rank-ordered 
national averages of principals’ perceptions of problematic cheating across schools within each nation 
[3]. There were many cheating cases that had become the findings of the authorities [4] and ironically, 
some of them did not only involve students but also teachers, even principals [5]. From time to time, 
the cases have attracted public concern and according to the Indonesian Teachers Union Federation 
[6], from 2013 to 2015 there were 1430 reports on national examination fraud throughout Indonesia. 
Following up on this issue, since 2015, the Indonesian Ministry of Education and Culture (MoEC) 
began to enact the computer-based national examination (CBNE). All NE’s results are later published 
in an open-access website annually along with the value of the national examination integrity index 
(NEI). 

The national examination integrity index (NEI) is the level of trust for school carried out on the 
national examination. It means that students are having not only the NE scores, but other concerned 
parties could also compare to the honesty value of NE Implementation [7]. To get the NEII score, the 
MoEC administered questionnaires for principals, teachers, and students in the schools [8]. The 
obtained scores were then published on the MoEC’s official website along with other statistics, trends, 
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and other calculations. The NEII scores range from 0-100 where the maximum value obtained when 
schools run CBNE. If a school reached 90 points of integrity index, this means that there was an 
indication (possibility) of 10% of fraud at the school [9]. According to the MoEC, [10] in a school 
with a high integrity index, the results are relatively the same, even when been tested with both paper 
or computer-based. 

In several publications, the MoEC stated that there was a declined national exam result at almost all 
levels of education (SMP, SMA, SMK), especially in mathematics and science subjects [11-12]. 
According to them [10], there are two possibilities that cause this trend; the complexity of the 
questions of the national examination, or the increasing integrity of national examination through the 
implementation of the CBNE. This study is intended to answer the second guess, in terms of the 
relationship between the national examination (NE) and the national examination integrity index 
(NEI), particularly for high school science classes in Aceh province. 

Based on the facts described above, there is a tendency by many parties that relates the trend 
between integrity index and national examinations score. This research, therefore, attends to answer 
such following questions: 

1. What are the trends and patterns showed from the pre-post CBNE implementation of CBNE at 
high school science classes in Aceh Province? 

2. How are the forms of relationship between the national examination (NE) and national 

examination integrity index (NEI) of high school science classes in Aceh Province? 


2. Methodology 

This research is a descriptive analysis study with an ex-post facto design. Data gathered in the form of 
NE’s and NEII’s value of 103,273 science classes students from 645 high schools at all 
regencies/cities in Aceh Province which published at the MoEC’s official website. The data were later 
reduced into a group of the average value (mean) per regencies/cities (23). 

The data obtained is then analyzed to get patterns, trends, and the forms of relationship between 
national examination (NE) scores and national examination integrity index (NEI) on pre and post 
(2015-2017) computer-based national examination (CBNE) of high school science classes in Aceh 
Province. To look at the trends and patterns of the pre and post CBNE implementation, descriptive 
statistics were used by comparing the average (mean) of the NE and NEII, the standard deviation of 
national examination (SDNE) in the group of regencies/cities. To get the picture of the relationship 
between the NEII and NE and, linear regression and curve estimation were applied. 


3. Results of the study 


3.1. Trends and patterns of the Pre-CBNE implementation 


Table 1. The average value (x) of national examination (NE), the standard deviation of 
national examination (SDNE), and national examination integrity index (NEI) of high 
school science students in all regencies of Aceh Province at the year 2015 (pre-CBNE) 


Regency x NE xX SDNE  xNEII Regency xNE  xSDNE _ x NEII 


1 74.83 0.6 38.4 13 74.8 6.9 54.0 
2 75.0 1.8 36.1 14 74.8 6.9 54.0 
3 74.3 2.5 36.0 15 72.1 7.2 59.7 
4 73.9 2.7 38.1 16 65.5 8.7 61.3 
5 70.4 3.4 37.0 17 59.0 9.8 55.4 
6 65.7 4.4 54.1 18 66.0 12.7 51.9 
7 74.6 4.7 38.0 19 60.1 13.2 51.2 
8 70.5 4.9 33.4 20 66.8 13.7 46.9 
9 67.3 5.8 41.9 21 60.7 14.8 47.1 


— 
oO 


67.6 6.1 43.5 22 57.3 16.2 49.8 
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Regency x NE XSDNE  xNEII Regency xNE  xSDNE _ xNEII 
11 70.2 6.6 38.6 23 47.53 16.9 64.9 
12 63.5 6.6 44.6 


Table 1 shows the result of national examination which estimated represent a similar value of how 
national examination undertook before CBNE was initiated in the province. The 23 regencies are later 
automatically ranked based on the average values sorted from the lowest to highest SDNE to magnify 
the direction of analysis. By sorting like this, the table turns out several patterns. The NEII average 
score ranges from 33.4 to 64.9. The number means that all science high schools were estimated to be 
35.1-66.6% cheating. In general, two-third (69%) of regencies in Aceh Province in this year have 
integrity index below 50%, while the rest is only 15% above. Compared to the interval interpretation 
below, 9 out of 23 regencies are strongly estimated to conduct a fraudulent performance in the 
examination. 
Tabel 2. The interval interpretation of national examination integrity index (NEI)! 


NEI Interval Cheating Estimation (%) Level of relationship 
(%) 
0-19,9 100-80,1 Very strong 
20-39,9 80-60, 1 Strong 
40-59,9 60-40, 1 Middle 


‘Such interval interpretation is required to clarify the data from the result of the study. 
The Interval Interpretation of the national examination integrity index (NEI) has never 
been revealed in other literature. The reliability of this interpretation is possibly gained 
further discussion. 

Specifically, in this pre-CBNE period, some regencies/cities in Aceh Province revealed striking 
figures. Compared to all regencies/cities, Regency | and 2 reached the highest xNE scores. However, 
they also possess the opposite integrity index which is one of the lowest. One surprising fact is that of 
all high school science students in Regency 1 occupied the highest NE ranks of all regencies/cities in 
Aceh, they had only 0.6 of x SDNE. This thin SDNE means that all science students in the regencies 
had almost similar NE scores (between 74.2-75.5). When we look closer to the number of students and 
schools, the Regency has 1,726 students of 29 number of education units. There are only two reasons 
behind the analysis, whether all 1,726 students possess almost similar competency or there was a 
massive and structured fraudulent practice of national examination in the regency. Inferring the 
analysis, the main pattern showed on the pre-CBNE period is high xNE, low xNEII, and low xSDNE. 
It leads analysis to estimate that there was a fraudulent practice of national examination in the regency 
at the time. 

Table 3. The average value (x) of national examination (NE), standard deviation of 
national examination (SDNE), and national examination integrity index (NEIJ) of high 
school science students in all regencies of Aceh Province at the year 2016 (1* year of 
CBNE’s implementation) 


Regency xXNE xSDNE XNEI Regency xNE xSDNE~ xNEII 


1 69.2 2.6 53.3 13 58.0 5.8 52.2 
2 70.8 3.0 52.5 14 58.0 5.8 52.2 
3 47.1 12.7 68.5 15 34.2 6.9 69.5 
4 37.8 11.1 75.0 16 43.2 12.3 71.0 
5 52.8 13.9 56.6 17 40.5 11.7 67.2 
6 62.2 10.3 53.8 18 59.6 9.3 33.9 
7 48.4 11.6 56.0 19 48.9 6.4 60.7 
8 55.0 75 58.7 20 60.9 16.5 43.0 
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Regency xXNE xSDNE XNEI Regency xNE xSDNE~ xXNEII 


9 52.0 16.1 52.6 21 41.7 12.5 64.6 
10 58.2 13.4 48.6 22 47.2 14.6 59.2 
11 46.9 5.8 51.5 23 58.0 5.8 52.2 
12 38.2 10.5 71.8 


Compared to the previous year, there was a change in the value of x NE and x NEII in the first year 
of CBNE (2016). In general, the value of x NE decreased from 67.5 to 50.8, while the x NEII value 
increased from 46.7 to 61.8. 86% of all districts have 50% or above of integrity Index, yet only 3 
regencies/cities that have Integrity below 50% (bold Regencies No.10, 18 and 20). In contrast to the 
previous trend, the three regencies above are not too prominent in the average NE scores, although still 
relatively high. 

On the closer look, Regency 1 and 2 which were highlighted in 2015, reveal slight changes. The 
pattern observed in the previous year is almost similar in the year after. Compared to all regencies, 
Regency | and 2 still ranked as the highest NE score along with the smallest standard deviation. These 
changes indicate the implementation of CBNE relates to changed values. 


Table 4. The average value (x) of national examination (NE), the standard deviation of 
national examination (SDNE), and national examination integrity index (NEII) of high 
school science students in all regencies of Aceh Province at the year 2016 (2™ year of 
CBNE’s implementation) 


Regency XNE  xSDNE x NEI Regency x NE x SDNE x NEI 


1 34.39 9.9 80.65 13 56.9 9.3 65.1 
2 39.9 9.7 73.8 14 56.8 9.3 65.1 
3 37.8 4.5 100 15 34.7 6 82.9 
4 37 6.8 81.3 16 42.1 8.3 100 
5 40 a2 100 17 37.2 6.7 60.3 
6 36.2 3.8 100 18 44.1 9.3 73 
7 47.2 12.6 71 19 41.3 0.9 100 
8 43.2 11.2 735 20 41.6 6.8 100 
9 41.1 6.9 75 21 36.2 8.2 79.9 
10 42.1 9.8 76.7 22 40.3 10.2 77.6 
11 43.5 8.1 63.4 23 39.1 7.2 100 
12 34.9 7.3 80.7 


In 2017, there were important changes from the previous two years; in terms of the average value 
of NE and NEII. The value of x NE decreased from 67.5 to 50.8 and became 41.2 in the last year. 
While the value of xNEII increased from 46.7 to 61.8, and 89.3 at last. Only 2 out of 23 
regencies/cities which have an average NE score above 50. The rest range from 34.7 to 47.0. 

Furthermore, Regency | and 2 which became the main point in 2015 and 2016 maintain the 
change. In 2017, the values continued to reveal a similar pattern that occurred before the Regency that 
ranked the highest NE had the opposite value of integrity. In addition to this, there were different 
patterns from the previous one. In the Regency 19 has a perfect level of integrity (100% used CBNE) 
but it turns out to reach average NE value with a thin standard deviation (0.9), almost the same as the 
standard deviation of Regency | had in 2015. 


3.2. Relationship between the national examination integrity Index (NEI) and national examination 
(NE) of high school science students in Aceh province 

From the background explained above, this study assumes that there have been a declined average of 
national examination (NE) of science high school students in Aceh Province due to the 
implementation of CBNE which appears through the national examination integrity index. The second 
question of this study is to picture the relationship between the two variables in two ways; linear 
regression and curve estimation. 
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The linear regression was run through several steps. First, a normality test was applied to determine 
whether the relationship follows a normal Gaussian distribution or not. The normality test of 
Kolmogorov-Smimov (the KZS value of the NEII variable) shows 0.812 while the NE variable is 
1.173. Both are higher than 0.05. So, it can be concluded that the two variables are assumed to be 
normally distributed and parametric procedures can be applied. 

Second, the coefficient of determination test was used to show how much the independent variable 
can explain the dependent variable. The higher coefficient determination value means the stronger 
ability of the independent variable to explain variations in changes of the dependent variable. To 
analyze the influence of the NEII on the NE, linear regression is used to prove that there are influences 
between the two variables. 


Table 5. Model Summary 


Adjusted R Std. Error of the 
Model R R Square Square Estimate 


1 .667* 445 436 15.34545 
*Predictors: (Constant), NE 

The table explains the regression relationship between the dependent variable (NEI) and the 
independent (NE). The influence relationship of NEII toward NE ranged from 0.436 (smallest) to 
0.667 (largest). If we take the middle regression value (R’), then the value of regression (R) is equal to 
0.445. This means that NEII variable contributes a 44.5% effect on NE variables. However, the 
number may still contain disturbing values that might cause errors of counts. So, the most countable 
value of relationship shows on adjusted R? value (0.436). It means that the relationship between NEII 
and NE is counted to be at least 43.6%, with 84.7% of accuracy (due to the estimated standard error 
value that is 15,345). To interpret the strength of the relationship from both variables, such an 
interpretation of coefficient intervals shows in table 6 below. 

Table 6. Interval Coefficient Interpretation 


Interval coefficient Level relationships 
0.00-0.10 Very low (negligible) 
0.10-0.39 Low 
0.40-0.69 Middle 
0.70-0.89 Strong 
0.00-1.00 Very strong 


Source: Schober P, Boer C, and Schwarte LA [13] 


Based on the Interval coefficient interpretation, the determination of the national examination 
integrity index on the national examination of science students in the Aceh Province ranges in the 
category of middle relationship. 

Third, the test of linearity was used to measure how much the integrity of the national examination 
(NEI) influences the national examination (NE) scores. Table 7 shows the coefficient as follows: 


Table 7. Coefficients * 
Standardized 
Unstandardized Coefficients Coefficients Collinearity Statistics 
Model B Std. Error Beta t Sig. Tolerance VIF 
1 (Constant) 119.831 7.582 15.805 .000 
NE -1.013 138 -.667 -7.324 .000 1.000 1.000 


“Dependent Variable: NEII 


The coefficient table shows that the value of sig. <0.05 which means that the regression model is 
linear. The table also shows the negative direction of the regression on the Beta coefficient (both 
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standard and non-standard), meaning that the higher the NEII value, the lower (opposite) the NE score 
would be. The constant value of 119.831 indicates that if there is no NEII, then the NE value is 
119.831. The regression coefficient X is -1.013 refers to every addition of 1 NEII value, the NE value 
increases by -1.013. Referring to the regression coefficient analysis, the regression equation obtained 
is NEII = 119.831 - (1,013) (NE). Furthermore, the t value = -7.324 with a significance value of 0.001 
<0.05, means that there is a significant influence between the NEII variables on the NE variable. 

The last linear regression stage of this method is the use of Product Moment-Pearson Correlation to 
determine the degree of closeness of the relationship between two variables. the Pearson correlation 
was computed using SPSS at the table below: 


Table 8. Correlations 


NEII NE 
NEII Pearson Correlation 1 -.667" 
Sig. (2-tailed) .000 
N 69 69 
NE Pearson Correlation -.667"° 1 
Sig. (2-tailed) .000 
N 69 69 


“Correlation is significant at the 0.01 level (2-tailed) 


The output shows the correlation analysis between NEII and NE with Pearson correlation value is 
1. Such criteria lead to the relationship between both variables to be very strong. Moreover, the SPSS 
analysis also shows a significant correlation at the level of 0.01. Meaning that there is a 99.9% 
influence of NEII variable on the NE variable. 

Besides the linear regression test, curve estimation was also applied to get the second picture of the 
relationship between NEII and NE variables. Parameter estimation from SPSS was firstly gauged to 
show the most suitable model for the relationship between NEI and NE in the form of the curve. 


Table 9. Model Summary and Parameter Estimates 


Dependent Variable:NEII 
Model Summary Parameter Estimates 
Equation R Square F dfl df2 Sig. Constant bl b2 b3 
Linear 445 53.646 67 000 119.831 -1.013 
Logarithmic 445 53.659 67 000 276.003 -53.286 


1 
1 
Inverse 434 51.369 = 1 67 000 12.786 2652.796 
Quadratic 447 26.701 2 66 .000 = 141.293 -1.850 —.008 
Cubic 448 26.788 2 66 000 136.342 -1.502 .000 5.303E-5 
Compound 453 55.447 1 67 .000 147.909 .984 
The independent variable is NE 


Table 9 shows that each model has the least difference in the R Square value compared to others. 
Among the six estimated patterns, the equation value of the compound model is the highest (0.453). 
This means that this model will give 45.3% an idea of the effectiveness of estimates. So that the 
estimation influences of NEII on NE is illustrated in the curve pattern below: 
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NEII 


© Observed 
— Compound 


Equation: ¥(NEID = 119.831 - (1.013) (NE) 


Figure 1. Curve Estimation Model of Relationship between NEII and NE 
Figure 1 depicts a two-dimensional relationship between the NE and NEII. This figure is an 
estimation model that occurred in the relationship with an equation of NEII = 119.831 - (1.013) (NE). 
The equation shows the inverse (negative) relationship wherein each addition of a NEII’s value 
contributes to a negative value of the NE. However, the curve also shows that the negative relationship 
is not perfectly linear, because the relationship line is appeared to be a slight curved shape. 


4. Discussion 

In the span of 3 years (2015-2017) there was a downward trend in the NE along with the increased 
NEII of high school science students in all districts in Aceh Province. A pattern that emerged in this 
period - is low integrity creates high NE scores - is seen very clearly in the pre-CBNE Period. 
However, after the introduction of CBNE, the NEII then rose gradually and consequently lowered the 
NE’s score of science students in Aceh Province. What revealed as striking patter here is when schools 
in a regency have a low x NEII, a high x NE score, with a small SDNE, then it leads to a possibility of 
massive and systematic fraudulent practice of the NE at the regencies (seen in Districts 1 and 2 in 
2015). This pattern can be modeled as follows. 


Higher NE 


Lower 2 SDNE 


Lower # NEII Fraudulent practice 


estimation of NE 


Figure 2. Model of fraudulent practice estimation of national examination 

The estimation model of NE’s fraudulent practice means that an implementation of national 
examination might be a fraud when in certain area possesses a low xNEII, high xNE, and small 
xSDNE. In this model, the main point to indicate the fraudulence is the SDNE. However, one SDNE 
alone, in this case, cannot be used as a benchmark, because the situation also found in 2017 where the 
Regency 19 has a low SDNE (0.9) yet also a high Integration index (100%). Looking closer to this, 
later check to show that all national exams in the regency that was conducted using computer-based 
examination (CBNE). So the low national examination standard deviation with a high integrity index 
is reasonable due to the number of students taking the national exam which was is relatively small 
(220 students). 

The gradual increase of NEII value in Aceh Province is undeniably related to the implementation 
of CBNE which had to apply in stages to reach the whole region. The trend also occurred nationally 
where the Ministry found difficulties in equipping all schools with certain facilities to implement 
CBNE [14]. In the first year of the CBNE nationally initiated in 2015, the province of Aceh has not 
yet implemented it. The examination in this province only began in the second year (2016). That is 
why we found such a striking difference between those years and changed gradually afterward. Until 
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2017, of the 645 high schools that hold science classes, 368 (57% of) schools had implemented the 
CBNE, while other 277 schools still carry out the examination based on paper and pencils. 

Comparing the national examination between the Aceh province’s and nation’s scores, we can see 
that the trend of change has also been happening nationally both for science high school students and 
other majors as been seen at the table below. 


Tabel 10. Annual comparison of national examination scores between provincial and national trend 


Year x NE of Aceh Province x NE Nationally 

All Class Science Class All Class Science Class 
2014/2015 65.96 69.13 61.30 65.29 
2015/2016 49.14 49.72 54.79 56.85 
2016/2017 40.89 41.96 50.38 52.87 


Source: PuspendikKemdikbud [15] 


Referring to the regression analysis results; the output interpretations indicate that the integrity of 
the national examination has a moderate influence on national examination (with the regression value 
of R*: 0.445) of science students in Aceh Province. The relationship between them is linear with the 
opposite direction (negative value), meaning that the higher the integrity index of the national 
examination, the lower the national examination obtained by the students. Both conclusions are 
consistent with other results found by Dwiatmoko et al. [16]. With relative similar variables, their 
study indicates that there was an existence of a systematic pattern to produce identical achievements 
(values) in a certain educational evaluation context. The unanimous results are also similar to this 
result study, which however later “leaving crumbs” for us to indicate that there was a systematic 
intervention on these achievements (mainly pre-CBNE period). Back to both periods, the government's 
efforts to obtain the reliability of the national examination with CBNE seemed to have had a 
significant impact. 


5. Conclusions 

This research concludes that there are striking differences and gradual changes between the pre-and 
post-implementation of the computer-based national examination of high school science students in 
the province of Aceh. In the span of three years (2015-2017) there was a downward trend of national 
examination (NE) along with the implementation of CBNE which reveals through the increased value 
of national examination integrity index (NEI) at all regencies in Aceh Province. In the pre-CBNE 
period, a pattern appeared in the form of low xNEII, high xNE, and low xSDNE. The pattern indicates 
the existence of a fraudulent practice estimation of a national examination in the regencies/cities. In 
the post-CBNE period, the trend and pattern shifted backward (negative direction) wherein, the 
national examination score was getting smaller, along with the increased NEII. In addition, the 
implementation of CBNE - which reveals through the national examination integrity index — 
moderately influence (with regression value R2: 0.445) the national examination scores of the high 
school science students in Aceh Province. The relationship between them is linear with the opposite 
direction (negative value), meaning that the higher the Integrity Index of the national examination, the 
lower the national examination obtained by the student. 
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